Method and apparatus for introducing reference impairments into video signals

ABSTRACT

A method and an apparatus are provided for introducing reference impairments into pictures to be used in subjective tests on digital video signals, in compliance with ITU-T Recommendation P.930. The method generates blockiness distortion and/or edge busyness distortion. For the first distortion, areas close to picture edges are selected and, through a movement characteristics analysis, blocks are identified that belong to said areas and that are concerned only by the actual movement of the subject and, for blocks chosen among the identified ones, the luminance value is replaced by a value linked to the individual pixel luminance and to the average block luminance value. For the second distortion, echoes are applied that are randomly chosen in a prearranged set and that alternate at random.

The present invention deals with digital video signal processingsystems, and more particularly it deals with a method of and anapparatus for introducing reference impairments into pictures to be usedin subjective tests on digital video signals.

In the field of digital video signal transmission, researches areconstantly made for coding (or compression) algorithms that allowreducing as much as possible the amount of information to be transmittedwhile guaranteeing at the same time a good quality of the coded signal.Signal quality is generally evaluated through subjective tests, in whicha comparison is made between the sequence of processed pictures and areference sequence. In case of a reasonably high quality (simply as anexample, in case of coding with bit rate in the compressed sequence ofthe order of some Mbit/s), the reference sequence is replaced by theoriginal unprocessed sequence. In case of compression algorithms withvery low bit rates (always as an example, bit rates of the order of somekbit/s), that bring about a greater quality reduction with respect tothe original sequence, it is preferred to use, as reference pictures,pictures subjected to distortion instead of those of the originalsequence, so that the reference sequence quality is not exceedinglydifferent from that of the sequence to be evaluated.

In order to guarantee a repeatability of subjective tests in time and indifferent measuring centres, ITU-T Draft Recommendation P.930“Principles of a Reference Impairment System for Video” states a set ofimpairments to be introduced—individually or jointly—into a sequence ofpictures to generate a reference sequence. The above Recommendationcatalogues the main types of distortion that are observed on a videosignal subjected to digital processing, and provides a brief descriptionof the origin of such distortions.

More in detail, the distortions being taken into account in theRecommendation are as follows:

Blockiness (or block distortion): it is caused by coarse quantization ofthe spatial frequency components during the coding process and it isgenerally more visible in the smoothest picture areas located next tomoving edges;

Blurring: it is the reduction in sharpness of edges and in spatialdetail caused by the need of a trade off, in compression algorithms,between number of usable bits on the one hand and resolution provided bycode and motion representation on the other hand;

Edge Busyness (echoes or pixel replicas with time-varying positions andintensities): it is a distortion that is concentrated at object edges,and it is caused by the use of relatively coarse quantization levelswhen coding a block that includes both edges delimiting rather smoothareas and pixels with a quite different average luminance level;

Noise around moving edges (known as Mosquito noise): it is a noisesubstantially similar to the previous one, characterised by added movingartefacts or by blotchy patterns superimposed over the edges;

“Pepper and salt” effect or quantization noise: it is a typical noise ofthe digital conversion processes and its appearance is similar to thewell-known “snow effect” of the analogue television, though it isdistributed over the picture in a non-uniform way;

Jerkiness: as its name states, it consists of the perception of anoriginally smooth movement as a series of snapshots.

An Appendix to the Recommendation also discloses the way of simulatingthe above distortions as regards luminance, and includes a proposal toimplement a system to generate them. This known system operates asfollows:

blockiness: it is obtained by identifying picture edges, to recogniseareas where such distortion is more commonly visible; in these areas, acertain number of blocks of N×N pixels where the distortion must beintroduced are selected, through an analysis of the displacementcharacteristics with respect to the previous frame; to actuallyintroduce the distortion, the pixel luminance value in each selectedblock is then replaced by a new value obtained by adding a random valueranging between −2 and +2 to the average between the original pixelluminance and the average block luminance; every block keeps thisdistortion for a fixed number of frames (for example 15) to make thiseffect more easily perceivable;

blurring: it is implemented by applying, frame by frame, a low-passunidimensional digital filtering on each frame line and by multiplyingthe filtered samples by a normalisation coefficient, in order to recoverthe correct amplitude dynamics;

edge busyness: it is implemented through a bidimensional filtering withfilters having ripples in the passband amplitude response: the rippleamplitude determines the amplitude or intensity of the echoes that areadded to simulate distortion, the ripple frequency determines the echodisplacement with respect to the picture and the phase determines theecho polarity;

quantization noise: it is simulated by randomly choosing a pre-setnumber of pixels and by replacing the luminance value with a randomvalue within a range of allowed values;

mosquito noise: it is simulated by adding a random value, chosen withinpre-set a range, to the luminance value for each pixel classified asbelonging to edges and having an associated motion;

jerkiness: it is simulated by repeating a same frame for a certainnumber of times.

The tests carried out have proven that the system proposed in theAppendix to the Recommendation is not fully satisfactory—that is, itdoes not allow an easy distortion recognition—at least as regards edgebusyness and blockiness distortions.

According to the present invention, instead, a method and an apparatusare provided through which the above distortions are generated so thatthe perception thereof is made easier during the tests.

The features of the invention are disclosed in the following claims 1 to12, as regards the method, and in claims 13 to 15, as regards theapparatus.

For a better understanding reference is made to the enclosed drawings,in which:

FIG. 1 is an overall block diagram of a system for carrying outsubjective tests on video pictures, comprising a reference picturegenerating system operating according to the invention;

FIG. 2, divided into two parts designated as 2A and 2B, is a flow chartof the operations related to blockiness distortion introduction;

FIG. 3, divided into two parts designated as 3A and 3B, is a flow chartof the operations related to edge busyness distortion introduction.

With reference to FIG. 1, a system for carrying out subjective tests ondigital picture processing apparatus includes a source 1 that generatesa sequence of test pictures that are digitally converted in ananalogue-to-digital converter 2 and are then applied both to a testapparatus 3, for example a digital coder-decoder, and to a referencepicture generating system 4. The latter is adapted to introduce certaindistortions into a picture that are generally chosen among thoseintroduced by the processing. The outputs of blocks 3, 4 are applied toa digital-to-analogue converter 5 that supplies the sequencesreconstructed to a monitor 6 for comparison by a viewer. Converters 2and 5 can also be directly mutually connected so that it is possible touse the sequence issued by source 1 as a reference: this is possible ifprocessing apparatus 3 supplies a processed picture whose quality isrelatively high and comparable with the original picture quality. Shouldthe system not be able to operate in real time, the images supplied byblocks 3, 4 can be collected by means of a recorder and reconvertedafterwards into analogue form to be applied to the monitor.

As a principle, the reference picture generating system 4 can beschematised by a suitably programmed processing unit 7 (for example aconventional digital signal processor), associated on the one side witha first memory 8, in which one or more frames of the test sequence to beprocessed are temporarily stored and from which the processed frames aredownloaded, and on the other side with a second memory 9 storing thenecessary information (for example, digital filter coefficient files,echo files, operating parameters, etc.) in order the processing unit 7can perform the processing required to introduce the differentdistortions. A console 10 connected to processing unit 7 will allow anoperator to select a desired operating mode (for example individualintroduction of one or more distortions or sequential application, in apre-set order, of all admitted distortions), to set the operatingparameters for the individual distortions, to keep track of thedistortions introduced and of the values chosen for differentparameters, etc.

This type of system can be used not only to create standard testexecution conditions and standard distortion conditions, so as to allowcomparing tests carried out in different times and in differentlaboratories, but also to quantify to a certain extent the test resultsin terms of introduced distortion degree: in fact the operator cansuccessively introduce, on a standard sequence of test pictures,different distortions by modifying parameters for each one of them (thiscorresponds to the modification of the introduced distortion level) tilla reference image is obtained whose quality, according to the viewer,corresponds to that of the picture produced by apparatus 3, and canrecord information related to test execution.

For better clarity, hereinbelow it is supposed that the distortionsintroduced by system 4 are indeed those described in ITU-TRecommendation P.930. In this case, the operations related to theapplication of blurring, quantization noise and mosquito noisedistortions are substantially similar to those described in the Appendixto the Recommendation and briefly mentioned above. Generation ofblockiness and edge busyness distortions, though complying with theprinciples stated in the Recommendation, is instead obtained in a mannerthat is the subject matter of the invention. The invention also improvesjerkiness generation.

Blockiness distortion generation requires the following operations:

areas are identified where it is likely that distortions appear in thetest sequence due to processing in apparatus 3 under test;

through an analysis of movement characteristics, a certain number ofpixel blocks, to which distortion is to be applied, is selected withinthose areas;

distortion is actually applied by replacing the pixel luminance valuefor each selected block with a value computed by the algorithm.

For each selected block, distortion is applied for the current frame andfor a number of following frames, thereby creating a persistence effect.In a preferred embodiment of the invention, the persistence period isrelatively limited, for example less than 10 frames and more preferablybetween 1 and 5 frames.

More particularly, at each frame, the operations are as follows.

The first operation consists in dividing the frame into blocks of N×Npixels, where N is a parameter set by the operator through console 10.Then, before proceeding, all blocks to which distortion has been appliedin previous frames and for which the persistence time is not elapsed yetare removed, since those blocks must not be further distorted in ordernot to make the persistence effect disappear.

Afterwards, areas where distortion must be introduced are identified. Asstated above, this type of distortion generally appears in rather smoothareas located next to edges and affected by movement, and a first stepis therefore identifying the edges. This operation in turn includes twosubsequent steps. The first step consists in evaluating the edgelocation by applying the so-called horizontal and vertical Sobel filtersto the whole picture and then in performing an average of the results.Sobel filters are algorithms approximating luminance derivative alonghorizontal and vertical directions and thereby they allow recognisingareas, like picture edges, where luminance has a sudden variation withrespect to adjacent areas. These algorithms are well known in the art.Further details can be found in the book “Digital image processing” byR. C. Gonzalez and R. E. Woods, Addison—Wesley Ed., pages 197-200 and418-420. The result of such processing is then compared, in the secondstep, with an input threshold: pixels for which the processing resulthas given a value greater than this threshold are classified asbelonging to edges. As a result of the edge detection algorithm, a frameis generated where the luminance value is forced to a certain pre-setvalue for pixels classified as edge pixels, to 0 for the other ones.

This procedure is the same as used for edge detection when applyingquantization noise distortion, even if the threshold and the pre-setluminance value an be different.

Once having identified the edges, movement has to be recognised, bydistinguishing however between subject movement and camera movement, soas to apply distortion only to blocks being affected by an actualsubject movement. The number of blocks to which distortion has to beapplied in a frame and the number of frames in which distortion has tobe applied to the different blocks are further algorithm parameters, setby the operator. Before the actual movement identification, a firstblock selection is carried out, to remove those directly belonging tothe picture edge and to take into account only blocks that are adjacentto edges or at a distance therefrom. For this purpose, the number Nb ofpixels classified as edge pixels is computed for each block, and thisnumber is compared with three threshold values k1, k2 and k3 (withk1>k2>k3), that are also three parameters to be set by the operator. Thethresholds define an upper range (k2−k1) and a lower range (0−k3) ofvalues. The blocks concerned are those for which Nb meets the conditionk2<Nb≦k1 and those for which Nb meets the condition 0≦Nb 23 k3, that isblocks having a number of edge pixels falling within the upper range andthose having a number of edge pixels falling within the lower range. Itis immediate to see that comparison with the upper range of valuesallows identifying blocks next to edges and comparison with the lowerthreshold allows identifying blocks that are relatively far from theedges. Suitable values for the thresholds can be, in case of an 8×8pixel block, 40 to 64 for k1, 4 to 20 for k2 and 0 to 3 for k3.

To identify movement in the remaining blocks, the change in theso-called temporal information with respect to the homologous block inthe previous frame is computed. The temporal information, as known, isrepresented by the difference between homologous pixel luminance insubsequent frames and is an indicator denoting movement. The computationis repeated also with reference to a certain number of further blocksobtained by displacing the homologous block in the previous frame withina certain neighbourhood by steps of a desired number of pixels. Thisallows detecting picture movement and movement direction. For each blockin the current frame, the minimum temporal information value is selectedamong the computed ones, that is the one providing the best movementevaluation, and blocks are removed whose absolute temporal informationvalue is less than a further threshold. This is the condition allowingmovements represented by camera displacements to be recognised andneglected: those movements are generally slow translations,characterised therefore by limited variations. Also the neighbourhoodsize, the step amplitude and the temporal information threshold areparameters that can be set by the operator.

Once having detected thereby the blocks affected by the actual movement,they are ordered by decreasing temporal information: if a value m hasbeen set for the number of blocks to be distorted, ordering them bydecreasing temporal information will allow applying the distortion tothose with greater temporal information, that is those with greaterdisplacements, if it is not possible to apply the distortion to allblocks. In fact, in a given frame, a certain number n₀ of blocks can bedistorted, n₀ being the difference between the number m of blocks forwhich distortion is required and the number m_(p) of blocks alreadydistorted to create the persistence effect with respect to previousframes: therefore, if the number of blocks detected through the temporalinformation analysis does not exceed n₀, distortion is applied to allblocks, otherwise it is applied only to the first n₀ blocks.

For the actual distortion application, finally, for every block to bedistorted, the average luminance is computed within the block and theluminance for each pixel is replaced by a weighed mean between theaverage luminance and the actual luminance of the examined pixel; theweighed mean is obtained by applying to the two values respectiveweights which are algorithm parameters set by the operator.Advantageously, the weights to be assigned to the pixel luminance and toaverage block luminance are the 1-complement of each other, andtherefore it is enough to store the values for a single weighingcoefficient. Suitable values for a weighing coefficient are in the rangebetween 0.2 and 0.6. The value obtained through the weighed mean is thenstill modified by adding a random term with null mean value and with amaximum value that is a further algorithm parameter, set by theoperator.

FIGS. 2A, 2B schematically show the described algorithm. In theseFigures, i, j are the indexes for the examined blocks. Given what isstated above, the diagram is self-explaining.

As regards edge busyness distortion, when initialising the test, a setof echoes that can be added to the picture itself are stored for eachpicture. Each echo will be identified by its co-ordinates, that is bythe vertical and horizontal distance (expressed in number of pixels)from the picture edge, by an intensity (gain or attenuation) and by apersistence time. The operator can establish in the initialisation phasethe number of echoes to be applied. In case multiple echoes are applied,their persistence periods will be established in order to avoid thesimultaneous change of more than one echo in a given frame. In apreferred embodiment, a echo with given co-ordinates, randomly chosenwithin the preselected set, is applied together with the symmetrical onewith respect to the picture (that is, if the selected echo hasco-ordinates xk, yk, the echo having co-ordinates −xk, −yk is alsoapplied). The two echoes of the pair obviously have the samepersistence. At the end of the persistence period for the or a pair ofechoes, the co-ordinates for the or a new pair will be also randomlyselected.

The above-specified algorithm is shown in detail in the flow chart inFIGS. 3A and 3B. Therein, is the frame index, j the replica index, p1 .. . pk are the persistence times for the different replicas; k1 . . . knare the indexes for already used replicas. Given what is stated above,the diagram is self-explaining.

The novelty features provided by the invention as regards blockinessdistortion are as follows: a) the invention allows distinguishingbetween subject movement and camera movement, thereby avoidingdistortion introduction in the areas characterised by a rigid and slowtranslation, in which the camera movement is recognised; b) in order toapply the distortion, after the camera movement has been compensated,not all blocks close to the edges are taken into account, but a finerchoice is carried out, by considering two groups of blocks, the firstcomprising blocks with a number of edge pixels included within an upperrange and the second composed of blocks with a number of edge pixelsincluded within a lower range; c) when actually applying the distortion,the weights assigned to original pixel luminance and to average blockluminance are variable and mutually different.

As regards edge busyness distortion, an arbitrary number of echoes areadded and they can be placed in any position, randomly occur and canhave variable persistence. Tests carried out have proven that theseinnovations really make the introduced distortion better perceivable bya viewer.

A further improvement in perceptibility can be obtained also as regardsjerkiness; for this distortion, in addition to reducing frame frequencyby replacing a certain number of frames with as many repetitions of theprevious frame, in case of interleaved pictures the field alternationcan be removed, by replacing for example the odd field with the evenone.

Hereinabove, only distortions actually concerned by the invention havebeen described in detail, without examining their interactions with oneanother and with other possibly applied distortions. These interactionsanyway result from the above mentioned Recommendation, that alsoprovides indications related to the distortion application order andmode when more than one distortion is to be applied.

It is apparent that what is disclosed is provided as a non-limitingexample and that variations and modifications can be introduced withoutdeparting from the scope of the invention. In particular, even if onlydistortion application to luminance pixels has been discussed, similardistortion criteria can be used for chrominance.

What is claimed is:
 1. Method of introducing reference impairments intodigital video pictures, wherein at least a first distortion isintroduced into a test picture, said first distortion being representedby a value modification at least of luminance pixels of pixel blockslocated next to picture edges and affected by a movement of the picture,and wherein for said modification, in every picture frame: a) pictureedges are identified; b) areas adjacent to said edges and including saidpixel blocks are selected, and blocks potentially distortable areidentified among the blocks belonging to said areas through an analysisof movement characteristics; c) the pixel luminance value for each blockto which distortion is actually to be applied is replaced by a valuelinked to the individual pixel luminance and to the average blockluminance; characterised in that, for said identification of blocks thatare potentially distortable, blocks with a number of edge pixels lyingwithin an upper range and blocks with a number of edge pixels lyingwithin a lower range are detected and, for the blocks detected, at everyframe an indicator of displacement with respect to a group ofcorresponding blocks in a previous frame is determined; saiddisplacement indicator is compared with a first threshold, and thedistortion is applied only to blocks for which the displacementindicator is not less than said first threshold, to prevent applicationof distortion to blocks in which movement is only due to displacementsof a camera picking up the picture.
 2. Method according to claim 1,characterised in that said luminance value modification includesreplacing the luminance value for each pixel in a block to be distortedwith an average between the original pixel luminance value and anaverage block luminance value, said values being weighed with respectivevariable weights selected in a set stored during a method initialisationstep.
 3. Method according to claim 2, characterised in that the weightsfor the original pixel luminance value and for the average blockluminance value are different and mutually complementary.
 4. Methodaccording to claim 1, characterised in that the limits of said upper andlower ranges for the number of edge pixels are chosen within respectiveranges stored during a method initialisation step.
 5. Method accordingto claim 1, characterised in that distortion is applied to a maximumnumber of blocks which is chosen within a value range established duringthe method initialisation step and, for each distorted block, distortionis made to persist for a number of subsequent frames varying from blockto block, said number of frames being also established in said methodinitialisation step, a block being excluded from further distortionapplication for the whole persistence period.
 6. Method according toclaim 1, characterised in that, for the comparison with said firstthreshold, the minimum displacement indicator is chosen for the currentblock.
 7. Method according to claim 1, characterised in that blockswhose displacement indicator is not less than the threshold and forwhich the persistence period has not elapsed are ordered by decreasingdisplacement indicator values and, if the number of these blocks isgreater than the difference between the maximum number of blocks set andthe number of blocks for which the persistence period of distortionapplied in previous frames has not elapsed, distortion is applied tothose with higher displacement indicators, in a number equal to saiddifference.
 8. Method according to claim 1, wherein a second distortionis applied, in alternative or in addition to the first distortion, saidsecond distortion being represented by adding pixels that are located ata distance from the picture and can persist for a number of frames,characterised in that in a current frame at least a pair of pixels areintroduced, composed of a first pixel whose distance from the picture,both along a vertical picture co-ordinate and along a horizontal pictureco-ordinate, is randomly selected within a set of possible horizontaland vertical distances established in said method initialisation step,and of a second pixel, that is the symmetrical pixel to the first onewith respect to the picture.
 9. Method according to claim 8,characterised in that a single pair of pixels is introduced.
 10. Methodaccording to claim 8, characterised in that a plurality of pairs ofpixels are introduced, and each of them is made to persist for a numberof frames varying from pair to pair and established in said methodinitialisation step, in such a way that different pairs change indifferent times.
 11. Method according to claim 8, characterised in that,at the end of the persistence period for a pair of pixels, theco-ordinates for a new pair of pixels to be applied to following framesare randomly selected within the same set of horizontal and verticaldistances.
 12. Method according to claim 8, wherein a third distortionis applied, in alternative or in addition to the first and/or the seconddistortion, said third distortion being represented by a frame frequencyreduction of a test picture obtained by repeating the same frame one ormore times, characterised in that, in case said test picture is aninterleaved picture, the field alternation is removed, by repeatingtwice a same field.
 13. Apparatus for introducing reference impairmentsinto sequences of digital video pictures, comprising: first memory means(8) for the temporary storage of subsequent frames of a picture to bedistorted and of frames resulting from distortion application; aprocessing unit (7) that accesses said first memory means to readpicture parts, to identify areas where to introduce one or more of saiddistortions, to modify said areas as required by the specificdistortions, and to write the modified picture to be supplied to displaymeans (6); second memory means (9) arranged to store operatinginformation necessary to apply the distortions; and a control unit (10)arranged to select operating apparatus modes and values of parameters tobe used and to store execution modes and values of parameters beingused, characterised in that said processing unit (7) is programmed inorder to introduce into a sequence of pictures at least a firstdistortion, represented by a value modification at least of luminancepixels in a number of pixel blocks located next to picture edges andaffected by a movement of the picture, through the following operations:a) identification of picture edges in every frame; b) identification ofpixel blocks having a number of edges pixels lying within an upper valuerange and of pixel blocks having a number of edge pixels lying within alower value range; c) computation, for each pixel block identified instep b), of the displacement with respect to every block of a pre-setgroup of pixel blocks in a previous frame, generation of a respectivedisplacement indicator and detection of the minimum displacementindicator for the current block; d) selection of pixel blocks having aminimum displacement indicator higher than a threshold; e) ordering ofthe pixel blocks selected in step d) according to the displacementindicator value; f) computation of the maximum number of blocks to whichframe distortion can be applied; g) modification of the luminance valuefor the pixels in a block by replacing the original value with anaverage between said original value and an average block luminancevalue, said values being weighed with respective weighing factors, andaddition to said average of a random value selected within a range thatcan be pre-set by means of said control unit (10), the modificationoperations being carried out for all blocks selected in step d), iftheir number does not exceed said maximum number, or only for those withgreater displacement indicators, in a number equal to said maximumnumber, if the number of blocks selected in step d) exceeds said maximumnumber.
 14. Apparatus according to claim 13, characterised in that saidprocessing unit (7) is programmed in order to introduce into thesequence of pictures, in alternative or in addition to the firstdistortion, also a second distortion, represented by random addition ofpixels that are spaced from the picture edge by a horizontal andvertical distance selected within a pre-established set of possibledistances and that can persist for a period selected within apre-established set of possible persistence periods, the added pixelscomprising at least a pair of pixels whose distances from the picturehave the same absolute values and opposite sign.
 15. Apparatus accordingto claim 14, characterised in that said processing unit (7) isprogrammed in order to introduce into the sequence of pictures, inalternative or in addition to one or both of said first and seconddistortions, also a third distortion represented by a reduction of theframe frequency, and to suppress, when applying said third distortion tointerleaved images, the field alternation, by twice replicating a samefield.